In many large computer systems such as are found in banks, at credit card transaction processing centers, etc., huge amounts of data must be moved and stored. Typically, very large disk arrays are used to store the data and these disk arrays are connected to file servers. These type systems require of the server/disk connection high reliability, high speed, large throughput and large bandwidth since huge amounts of information are being processed and a system shutdown or slowdown adversely impacts customers (and possibly safety in the case of large air traffic control computer systems). Because these systems can be spread out over several buildings or even if all the servers and disk arrays are within the same building, the easiest way to interconnect all the servers and disk drives for maximum redundancy is through a local area network usually with a hub having drop lines connected to all servers, disk drives and other units. Because disks fail frequently and must be replaced by redundant disks, there is a large amount of management and control traffic that must go back and forth over the server/disk array connections and through the hub to the management and control process. This management and control traffic is necessary to determine which disks are on-line, where those disks are on the network, operational disk status, how many fans are still running (because when a fan fails, the disk it cools will probably be next), what servers are still operational, network fault status, network traffic conditions and statistics, etc.
Out of band management topologies get management data to and from the hub easily enough on separate network segments, which has its own set of problems described below. However, getting management and control data to and from the disk arrays in-band is more difficult. The management and control data sits in registers on a board in the disk array. In order to get this low speed, low priority data onto the high speed data path, special circuitry must be built in each disk array which interfaces these registers to the high speed data path. This circuitry functions to collect and format the management and control data into the type of data packets used in the high speed data path and to transmit these packets with the correct communication protocol to the hub. Since the management and control data does not consume an entire packet of the size used to send data on the high speed data path, some packet space is empty and wasted . The need for this special circuitry to put the management and control data in the high speed data path makes the disk arrays more expensive and complex and placing management and control data in the high speed data packets wastes network throughput.
Some disk arrays store the management and control data on a separate disk drive which can be polled. However, this approach does not solve the problem of the need for special circuitry to get the management and control data into packets in the high speed data path and the resulting extra complexity and wasted throughput. It only allows time shifting thereby enabling transmission of the management and control data when network traffic volume is low and the throughput loss is not as significant. This time shifting is implemented by providing more storage capacity for the management and control data than is provided in the registers of the other prior art type disk arrays described above.
In some networks, management traffic is transmitted in-band by placing the management and control packets inside empty data packets and shipping these partially filled packets over the existing network connections between the server, hub and disk drives. This causes loss in throughput since the data packets are large and the management data does rot fill the data packets entirely. This leads to wasted bandwidth. Further, arbitration by the management process to have one or more packets awarded to it for management traffic and to have access to the network so that transmissions to all servers and disk drive arrays of management messages consumes processing resources and network throughput unnecessarily.
One proposal has been made in the prior art to interleave special management and control packets in with the packet stream on the high speed data path to alleviate the above mentioned problem. However, this makes the design of the integrated circuits that implement the processing on the various network protocol layers more complex and difficult. So far this approach has been a commercial failure.
Another approach that has been tried in the prior art for transmission of management and control packets is to provide an entirely separate network for the management and control data such that each server and disk array is connected to the out of band management and control process running on a separate diagnostic processor by its own network segment. This substantially increases the wiring and connection cost of the system, especially in distributed systems, as every server and disk array must have an additional network segment connected thereto. In addition, each server and disk array uses a card slot for the management and control network card which adds to the expense, complexity and failure point count of each of these units.
A multiplexing approach that has been used in the prior art to send multiple television signals over the same media is represented by U.S. Pat. No. 3,623,105. This patent teaches receiving multiple video signals and translating each one to a different channel or subband and adding all the subband signals together to form a composite signal. The composite signal is then applied to the frequency control input of a VCO having a nominal frequency of 750 mHz. The output of the VCO is applied to one input of a pulse width modulator the other input of which receives an 18 gHz carrier. The pulse width modulator serves to key the 18 gHz carrier in accordance with the period of the signal from the VCO so that what is transmitted is a train of 18 gHz waves the width of each wave or "pulse" being set by the instantaneous period of the signal output from the VCO.
Another approach to implement a subchannel or auxiliary channel over a digital communication system is represented by U.S. Pat. No. 4,079,203 to Dragoo. This patent teaches an auxiliary channel implemented on a time division multiplexed carrier system by modulating the pulse repetition rate of the digital bitstream on the transmit side of the transaction. Each transceiver transmit section includes a FIFO shift register acting as a buffer. Modulation of the auxiliary channel data is carried out by varying the rate at which the digital information of the main channel is clocked out of the FIFO shift register at the transmitting end. The modulating signal varies the pulse repetition rate of a voltage controlled multivibrator which has its output coupled to the clock out input of the FIFO shift register and to a phase comparator of a phase lock loop. Data is clocked in using a clock signal derived from the incoming serial data stream. One drawback of this system is that the capacity of the FIFO can be exceeded if the clock out rate falls substantially behind the clock in rate because of the nature of the modulating signal during certain intervals.
Another approach which has been tried in the prior art is represented by U.S. Pat. No. 4,425,642 to Moses et al. This patent teaches sending digital data simultaneously with analog signals over the same media used by an analog signal communication system such as a telephone or television video. This is done by converting the digital data into very low power multifrequency signals consisting of fundamentals and harmonics. Although the harmonics are in the frequency range of the main signals, their low power does not cause substantial interference. Complicated filter arrangements at the receiver end separate out the harmonics and fundamentals that encode the digital signals and demodulation circuitry at the receiver decodes these frequencies back into digital data. This approach is not well suited to a system where the main data flow is digital and requires complex filtering and decoding arrangements at the receiver and is therefore unduly expensive and complex.
Another approach that has been tried in the prior art is exemplified by U.S. Pat. No. 4,677,608 to Forsberg. This patent teaches a method of implementing a service channel over a fiber optic system line which carries high data rate, e.g., 34 megabits/sec, data encoded with a code the power spectrum of which is heavily suppressed for low frequencies. The low frequency section of the media bandwidth is used to transmit a service channel with a low bandwidth. The service channel signals are frequency modulated, and this frequency modulated signal carrying the service channel data is then used to pulse width modulate pulses forming the high speed data stream.
It is critical to note in fully understanding the invention that this Forsberg scheme only transmits high speed data in the RZ or return to zero format on the fiber optic media, and, if NRZ format high speed data is input to the pulse width modulator, the NRZ format high speed data is converted to RZ format high speed data for transmission on the fiber optic media by performing an AND operation with the clock, as shown in FIG. 5 (see Col. 3, lines 58-61). Either way, the RZ data is a stream of pulses (during logic 1 bit times cnly) that are pulse width modulated by the subchannel carrier signal. RZ format high speed data is not compatible with standard high speed data receivers designed in accordance with this Fibre Channel and Gigabit Ethernet standards which are the network environments in which the various species of the invention disclosed herein are intended to work. Also, with RZ data, the number of transitions from one logic state to another is approximately double the number of transitions of NRZ data over the same number of bit times. Forsberg's system pulse width modulates each pulse representing a logic 1 so he gets perturbations of the lime of crossing a reference voltage for both a rising edge and a falling edge of that pulse. With RZ data, if two consecutive logic 1's are followed by two consecutive logic 0's over four bit times, there will be a pulse with two transitions in each of the first two bit times (representing two consecutive logic 1's) and no pulses in the next two bit times. In contrast, with NRZ data for the same hypothetical, there will be a rising edge at the beginning of the first bit time and a falling edge at the end of the second bit time and no transitions during the third and fourth bit times. The critical distinction is that NRZ data has a transition density which is variable over time depending upon the content of the data while RZ data does not have a variable transition density. Since the subchannel information in Forsberg's system is expressed in the form of perturbations of the time of crossing the reference voltage on each transition of all logic 1 pulses, his system has a know and reliable number of transitions in which to send subchannel energy. Therefore, Forsberg does not have a variable transition density probleme, and teaches no circuitry to deal with the fact that where pulse width modulation is in used to carry the subchannel information, transition density variations vary the magnitude of the subchannel signal at the receiver. Because RZ data is used exclusively in the Forsberg system as the data format on the media, a much stronger and nonvarying amplitude subchannel signal is available at the subchannel receiver which is easier to detect in the presence of noise is availabe.
Forsberg's system apparently was designed to work in telephony systems built by Ericsson (the assignee of the Forsberg technology) which are not transmitting One Gigabit digital data on a local area network. This conclusion can be drawn from the teachings at Col. 2. line 65 where the subchannel data rate is indicated to be 0.3-4 kHz, and Col. 3, line 6 where the FM modulated subchannel data signal is indicated to be 35-65 kHz, and from Col. 2, line 54 where the high speed bit rate is taught to be f.sub.0 and Col. 3, lines 21 and 22 where the clock rate is taught to be f.sub.0 =1/T and Col. 3, lines 35-36 where the clock frequency is taught to be around 40 MHz.
Therefore, a need has arisen for a method and apparatus of implementing a subchannel which shares the same media as the high speed data channel and which can be used regardless of whether the media is fiber optic or copper and which can be used when the high speed data is in NRZ format with a variable transition density. The desired product will be one over which subchannel data is sent over the shared media out of band thereby not adversely impacting throughput of the network for the high speed data. Further, the preferred form of the desired product will be compatible with existing high speed data receivers and line transceivers so that no modifications thereto are required and so that the presence of subchannel data Fourier components in the high speed data signal will not increase bit error rates of the conventional high speed data receivers. Also, the desired product will not unduly complicate the circuitry at both ends of the media and will not be jammed by the high speed data transmissions.